Driving pattern recognition and safety control

ABSTRACT

Systems and methods are provided for controlling a vehicle. A safe envelope driving pattern is determined to control the vehicle in an autonomous mode. User identification data and sensor data are received from one or more sensors associated with the vehicle. A driver-specific driving pattern is determined based on the received sensor data and the user identification data. Operation of the vehicle is controlled in the autonomous mode based on the identification of the user in the driver&#39;s seat, the safe envelope driving pattern, and the user-specific driving pattern.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of the filing dates of U.S.Provisional Application No. 61/390,094, entitled “AUTONOMOUS VEHICLES,”filed Oct. 5, 2010, and U.S. Provisional Application No. 61/391,271,entitled “AUTONOMOUS VEHICLES,” filed Oct. 8, 2010, the entiredisclosures of which are hereby incorporated herein by reference.

BACKGROUND

The present disclosure relates generally to safe and effective use ofautonomous vehicles. More particularly, aspects of the disclosure relateto learning a driver's driving behavior and controlling the autonomousdriving mode based on the learned behavior and predetermined safedriving patterns.

Autonomous vehicles may be configured to be driven in a manual mode(where the operator exercises a high degree of control over the movementof the vehicle) or in an autonomous mode (where the vehicle essentiallydrives itself). These vehicles use various computing systems to aid inthe transport of passengers from one location to another. Someautonomous vehicles may require some initial input or continuous inputfrom an operator, such as a pilot, driver, or passenger. Other systems,for example autopilot systems, may be used only when the system has beenengaged, which permits the operator to switch from a manual mode to anautonomous mode and to other modes that lie somewhere in between.

A vehicle with autonomous driving capability may be adapted to recognizethe type of driver, such as a teenage driver or an older driver, andmodify the driving parameters in response to the recognition. Forexample, driving speed may be limited for a teenage driver and lessjerky movements may be obtained by the autonomous vehicle for an olderdriver.

SUMMARY

Aspects of the disclosure provide systems and methods for determining adriver-specific driving pattern and controlling an autonomous vehiclebased on the driver-specific pattern and a predetermined safe envelopedriving pattern. By learning a specific driver's driving behavior andadapting an autonomous driving mode to the driver's style, an autonomousvehicle may be configured with some general safe envelope drivingpatterns. The adaptive autonomous driving allows the vehicle to performautonomous driving within the general safe envelope but still safely inthe driver's preferred style of driving.

The autonomous vehicle may monitor the vehicle's movement, and collectdata such as speed, lane changes, and changes in space between vehiclesin front of and behind the driver. The vehicle may also collect datasuch as (but not limited to) gas usage and application ofthrottles/brakes. Various driver preferences may also be recorded orderived from the collected data. Driver preferences may include apreference of windy roads over straight roads, right turns over leftturns, multi-lane highways over side roads, etc. From data collectedover time, the vehicle may learn the driver's driving pattern and gaugedriver preferences. Various machine learning algorithms may be used tofacilitate the learning process. The learning period may depend onvarious factors such as the specific driving feature that the vehicletries to learn (e.g., it may take a shorter time to learn a preferredspeed of acceleration than a preference of multi-lane highways over sideroads).

When the vehicle learns the driving patterns, the vehicle may performdriver-specific autonomous driving when it identifies which driver iscurrently in the driver's seat (e.g., changing lanes more often to movefaster for driver A than for driver B, selecting a route with more windyroads for driver C, and selecting a route with more straight roads fordriver D). The machine-learned driving pattern may be manuallyinterrupted by the driver when desired.

The data may be collected by various types of sensor systems on thevehicle. These sensor systems may include GPS, inertial sensors, lasers,radar, sonar, and acoustic sensors. Other feedback signals, such aswheel speeds and throttle/brake pressure, may also be read from thevehicle. Data processing may take place at the vehicle or transmittedexternal to the vehicle and performed remotely.

In one aspect, a method for controlling a vehicle includes determining,by a processor, a first driving pattern to control the vehicle in anautonomous mode. Sensor data is received from one or more sensorsassociated with the vehicle. The processor determines a second drivingpattern based on the received sensor data and received useridentification data. A user in a driver's seat of the vehicle isidentified. The processor controls operation of the vehicle in theautonomous mode based on the identified user, the second pattern, andthe first pattern.

In another aspect, a device for controlling a vehicle with an autonomousdriving mode includes one or more sensors for detecting informationsurrounding the vehicle, a processor coupled to the one or more sensors,and a memory coupled to the processor and storing instructionsexecutable by the processor causing the processor to: determine a firstdriving pattern to control the vehicle in an autonomous mode; receiveuser identification data; receive sensor data from one or more sensorsassociated with the vehicle; determine a second driving pattern based onthe received sensor data and the received user identification data;identify a user in a driver's seat of the vehicle; and control operationof the vehicle in the autonomous mode based on the identified user, thesecond pattern, and the first pattern.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional diagram of a system in accordance with anexemplary embodiment.

FIG. 2 is an interior of an autonomous vehicle in accordance with anexemplary embodiment.

FIG. 3 is a system diagram of an autonomous vehicle driving system inaccordance with an exemplary embodiment.

FIG. 4 is a flow diagram for management of an autonomous vehicle inaccordance with an exemplary embodiment.

FIG. 5 is another flow diagram in accordance with an exemplaryembodiment.

DETAILED DESCRIPTION

Aspects, features and advantages of the disclosure will be appreciatedwhen considered with reference to the following description of exemplaryembodiments and accompanying figures. The same reference numbers indifferent drawings may identify the same or similar elements.Furthermore, the following description is not limiting; the scope of thedisclosure is defined by the appended claims and equivalents.

As shown in FIG. 1, an autonomous driving system 100 in accordance withone aspect includes a vehicle 101 with various components. While certainaspects are particularly useful in connection with specific types ofvehicles, the vehicle may be any type of vehicle including, but notlimited to, cars, trucks, motorcycles, busses, boats, airplanes,helicopters, lawnmowers, recreational vehicles, amusement park vehicles,construction vehicles, farm equipment, trams, golf carts, trains, andtrolleys. The vehicle may have one or more computers, such as computer110 containing a processor 120, memory 130 and other componentstypically present in general purpose computers.

The memory 130 stores information accessible by processor 120, includinginstructions 132 and data 134 that may be executed or otherwise used bythe processor 120. The memory 130 may be of any type capable of storinginformation accessible by the processor, including a computer-readablemedium, or other medium that stores data that may be read with the aidof an electronic device, such as a hard-drive, memory card, ROM, RAM,DVD or other optical disks, as well as other write-capable and read-onlymemories. Systems and methods may include different combinations of theforegoing, whereby different portions of the instructions and data arestored on different types of media.

The instructions 132 may be any set of instructions to be executeddirectly (such as machine code) or indirectly (such as scripts) by theprocessor. For example, the instructions may be stored as computer codeon the computer-readable medium. In that regard, the terms“instructions” and “programs” may be used interchangeably herein. Theinstructions may be stored in object code format for direct processingby the processor, or in any other computer language including scripts orcollections of independent source code modules that are interpreted ondemand or compiled in advance. Functions, methods and routines of theinstructions are explained in more detail below.

The data 134 may be retrieved, stored or modified by processor 120 inaccordance with the instructions 132. For instance, although the systemand method is not limited by any particular data structure, the data maybe stored in computer registers, in a relational database as a tablehaving a plurality of different fields and records, XML documents orflat files. The data may also be formatted in any computer-readableformat. By further way of example only, image data may be stored asbitmaps comprised of grids of pixels that are stored in accordance withformats that are compressed or uncompressed, lossless (e.g., BMP) orlossy (e.g., JPEG), and bitmap or vector-based (e.g., SVG), as well ascomputer instructions for drawing graphics. The data may comprise anyinformation sufficient to identify the relevant information, such asnumbers, descriptive text, proprietary codes, references to data storedin other areas of the same memory or different memories (including othernetwork locations) or information that is used by a function tocalculate the relevant data.

The processor 120 may be any conventional processor, such as commercialCPUs. Alternatively, the processor may be a dedicated device such as anASIC. Although FIG. 1 functionally illustrates the processor, memory,and other elements of computer 110 as being within the same block, itwill be understood by those of ordinary skill in the art that theprocessor and memory may actually comprise multiple processors andmemories that may or may not be stored within the same physical housing.For example, memory may be a hard drive or other storage media locatedin a housing different from that of computer 110. Accordingly,references to a processor or computer will be understood to includereferences to a collection of processors, computers or memories that mayor may not operate in parallel. Rather than using a single processor toperform the steps described herein some of the components such assteering components and deceleration components may each have their ownprocessor that only performs calculations related to the component'sspecific function.

In various aspects described herein, the processor may be locatedremotely from the vehicle and communicate with the vehicle wirelessly.In other aspects, some of the processes described herein are executed ona processor disposed within the vehicle and others by a remoteprocessor, including taking the steps necessary to execute a singlemaneuver.

Computer 110 may include all of the components normally used inconnection with a computer such as a central processing unit (CPU),memory (e.g., RAM and internal hard drives) storing data 134 andinstructions such as a web browser, an electronic display 142 (e.g., amonitor having a screen, a small LCD touch-screen or any otherelectrical device that is operable to display information), user input(e.g., a mouse, keyboard, touch screen and/or microphone), as well asvarious sensors (e.g. a video camera) for gathering the explicit (e.g.,a gesture) or implicit (e.g., “the person is asleep”) information aboutthe states and desires of a person.

The vehicle may also include a geographic position component 144 incommunication with computer 110 for determining the geographic locationof the device. For example, the position component may include a GPSreceiver to determine the device's latitude, longitude and/or altitudeposition. Other location systems such as laser-based localizationsystems, inertia-aided GPS, or camera-based localization may also beused to identify the location of the vehicle. The vehicle may alsoreceive location information from various sources and combine thisinformation using various filters to identify a “best” estimate of thevehicle's location. For example, the vehicle may identify a number oflocation estimates including a map location, a GPS location, and anestimation of the vehicle's current location based on its change overtime from a previous location. This information may be combined togetherto identify a highly accurate estimate of the vehicle's location. The“location” of the vehicle as discussed herein may include an absolutegeographical location, such as latitude, longitude, and altitude as wellas relative location information, such as location relative to othercars in the vicinity which can often be determined with less noise thanabsolute geographical location.

The device may also include other features in communication withcomputer 110, such as an accelerometer, gyroscope or anotherdirection/speed detection device 146 to determine the direction andspeed of the vehicle or changes thereto. By way of example only, device146 may determine its pitch, yaw or roll (or changes thereto) relativeto the direction of gravity or a plane perpendicular thereto. The devicemay also track increases or decreases in speed and the direction of suchchanges. The device's provision of location and orientation data as setforth herein may be provided automatically to the user, computer 110,other computers and combinations of the foregoing.

The computer may control the direction and speed of the vehicle bycontrolling various components. By way of example, if the vehicle isoperating in a completely autonomous mode, computer 110 may cause thevehicle to accelerate (e.g., by increasing fuel or other energy providedto the engine), decelerate (e.g., by decreasing the fuel supplied to theengine or by applying brakes) and change direction (e.g., by turning thefront wheels).

The vehicle may include components 148 for detecting objects external tothe vehicle such as other vehicles, obstacles in the roadway, trafficsignals, signs, trees, etc. The detection system may include lasers,sonar, radar, cameras or any other detection devices. For example, ifthe vehicle is a small passenger car, the car may include a lasermounted on the roof or other convenient location. In one aspect, thelaser may measure the distance between the vehicle and the objectsurfaces facing the vehicle by spinning on its axis and changing itspitch. The laser may also be used to identify lane lines, for example,by distinguishing between the amount of light reflected or absorbed bythe dark roadway and light lane lines. The vehicle may also includevarious radar detection units, such as those used for adaptive cruisecontrol systems. The radar detection units may be located on the frontand back of the car as well as on either side of the front bumper. Inanother example, a variety of cameras may be mounted on the car atdistances from one another which are known so that the parallax from thedifferent images may be used to compute the distance to various objectswhich are captured by two or more cameras. These sensors allow thevehicle to understand and potentially respond to its environment inorder to maximize safety for passengers as well as objects or people inthe environment.

In addition to the sensors described above, the computer may also useinput from sensors typical of non-autonomous vehicles. For example,these sensors may include tire pressure sensors, engine temperaturesensors, brake heat sensors, brake pad status sensors, tire treadsensors, fuel sensors, oil level and quality sensors, air qualitysensors (for detecting temperature, humidity, or particulates in theair), etc.

Many of these sensors provide data that is processed by the computer inreal-time; that is, the sensors may continuously update their output toreflect the environment being sensed at or over a range of time, andcontinuously or as-demanded provide that updated output to the computerso that the computer can determine whether the vehicle's then-currentdirection or speed should be modified in response to the sensedenvironment.

These sensors may be used to identify, track and predict the movementsof pedestrians, bicycles, other vehicles, or objects in the roadway. Forexample, the sensors may provide the location and shape information ofobjects surrounding the vehicle to computer 110, which in turn mayidentify the object as another vehicle. The object's current movementmay be also be determined by the sensor (e.g., the component is aself-contained speed radar detector), or by the computer 110, based oninformation provided by the sensors (e.g., by comparing changes in theobject's position data over time).

The computer may change the vehicle's current path and speed based onthe presence of detected objects. For example, the vehicle mayautomatically slow down if its current speed is 50 mph and it detects,by using its cameras and using optical-character recognition, that itwill shortly pass a sign indicating that the speed limit is 35 mph.Similarly, if the computer determines that an object is obstructing theintended path of the vehicle, it may maneuver the vehicle around theobstruction.

The vehicle's computer system may predict a detected object's expectedmovement. The computer system 110 may simply predict the object's futuremovement based solely on the object's instant direction,acceleration/deceleration and velocity, e.g., that the object's currentdirection and movement will continue.

Once an object is detected, the system may determine the type of theobject, for example, a traffic cone, person, car, truck or bicycle, anduse this information to predict the object's future behavior. Forexample, the vehicle may determine an object's type based on one or moreof the shape of the object as determined by a laser, the size and speedof the object based on radar, or by pattern matching based on cameraimages. Objects may also be identified by using an object classifierwhich may consider one or more of the size of an object (bicycles arelarger than a breadbox and smaller than a car), the speed of the object(bicycles do not tend to go faster than 40 miles per hour or slower than0.1 miles per hour), the heat coming from the bicycle (bicycles tend tohave a rider that emits body heat), etc.

In some examples, objects identified by the vehicle may not actuallyrequire the vehicle to alter its course. For example, during asandstorm, the vehicle may detect the sand as one or more objects, butneed not alter its trajectory, though it may slow or stop itself forsafety reasons.

In another example, the scene external to the vehicle need not besegmented from input of the various sensors, nor do objects need to beclassified for the vehicle to take a responsive action. Rather, thevehicle may take one or more actions based on the color and/or shape ofan object.

The system may also rely on information that is independent of thedetected object's movement to predict the object's next action. By wayof example, if the vehicle determines that another object is a bicyclethat is beginning to ascend a steep hill in front of the vehicle, thecomputer may predict that the bicycle will soon slow down—and will slowthe vehicle down accordingly—regardless of whether the bicycle iscurrently traveling at a relatively high speed.

It will be understood that the foregoing methods of identifying,classifying, and reacting to objects external to the vehicle may be usedalone or in any combination in order to increase the likelihood ofavoiding a collision.

By way of further example, the system may determine that an object nearthe vehicle is another car in a turn-only lane (e.g., by analyzing imagedata that captures the other car, the lane the other car is in, and apainted left-turn arrow in the lane). In that regard, the system maypredict that the other car may turn at the next intersection.

The computer may cause the vehicle to take particular actions inresponse to the predicted actions of the surrounding objects. Forexample, if the computer 110 determines that another car approaching thevehicle is turning, for example based on the car's turn signal or inwhich lane the car is, at the next intersection as noted above, thecomputer may slow the vehicle down as it approaches the intersection. Inthis regard, the predicted behavior of other objects is based not onlyon the type of object and its current trajectory, but also based on somelikelihood that the object may or may not obey traffic rules orpre-determined behaviors. This may allow the vehicle not only to respondto legal and predictable behaviors, but also correct for unexpectedbehaviors by other drivers, such as illegal u-turns or lane changes,running red lights, etc.

In another example, the system may include a library of rules aboutobject performance in various situations. For example, a car in aleft-most lane that has a left-turn arrow mounted on the light will verylikely turn left when the arrow turns green. The library may be builtmanually, or by the vehicle's observation of other vehicles (autonomousor not) on the roadway. The library may begin as a human-built set ofrules which may be improved by vehicle observations. Similarly, thelibrary may begin as rules learned from vehicle observation and havehumans examine the rules and improve them manually. This observation andlearning may be accomplished by, for example, tools and techniques ofmachine learning.

In addition to processing data provided by the various sensors, thecomputer may rely on environmental data that was obtained at a previouspoint in time and is expected to persist regardless of the vehicle'spresence in the environment. For example, data 134 may include detailedmap information 136, for example, highly detailed maps identifying theshape and elevation of roadways, lane lines, intersections, crosswalks,speed limits, traffic signals, buildings, signs, real time trafficinformation, or other such objects and information. Each of theseobjects such as lane lines or intersections may be associated with ageographic location which is highly accurate, for example, to 15 cm oreven 1 cm. The map information may also include, for example, explicitspeed limit information associated with various roadway segments. Thespeed limit data may be entered manually or scanned from previouslytaken images of a speed limit sign using, for example, optical-characterrecognition. The map information may include three-dimensional terrainmaps incorporating one or more of objects listed above. For example, thevehicle may determine that another car is expected to turn based onreal-time data (e.g., using its sensors to determine the current GPSposition of another car) and other data (e.g., comparing the GPSposition with previously-stored lane-specific map data to determinewhether the other car is within a turn lane).

In another example, the vehicle may use the map information tosupplement the sensor data in order to better identify the location,attributes, and state of the roadway. For example, if the lane lines ofthe roadway have disappeared through wear, the vehicle may anticipatethe location of the lane lines based on the map information rather thanrelying only on the sensor data.

The vehicle sensors may also be used to collect and supplement mapinformation. For example, the driver may drive the vehicle in anon-autonomous mode in order to detect and store various types of mapinformation, such as the location of roadways, lane lines,intersections, traffic signals, etc. Later, the vehicle may use thestored information to maneuver the vehicle. In another example, if thevehicle detects or observes environmental changes, such as a bridgemoving a few centimeters over time, a new traffic pattern at anintersection, or if the roadway has been paved and the lane lines havemoved, this information may not only be detected by the vehicle and usedto make various determination about how to maneuver the vehicle to avoida collision, but may also be incorporated into the vehicle's mapinformation. In some examples, the driver may optionally select toreport the changed information to a central map database to be used byother autonomous vehicles by transmitting wirelessly to a remote server.In response, the server may update the database and make any changesavailable to other autonomous vehicles, for example, by transmitting theinformation automatically or by making available downloadable updates.Thus, environmental changes may be updated to a large number of vehiclesfrom the remote server.

In another example, autonomous vehicles may be equipped with cameras forcapturing street level images of roadways or objects along roadways.

Computer 110 may also control status indicators 138, in order to conveythe status of the vehicle and its components to a passenger of vehicle101. For example, vehicle 101 may be equipped with a display 225, asshown in FIG. 2, for displaying information relating to the overallstatus of the vehicle, particular sensors, or computer 110 inparticular. The display 225 may include computer generated images of thevehicle's surroundings including, for example, the status of thecomputer, the vehicle itself, roadways, intersections, as well as otherobjects and information.

Computer 110 may use visual or audible cues to indicate whether computer110 is obtaining valid data from the various sensors, whether thecomputer is partially or completely controlling the direction or speedof the car or both, whether there are any errors, etc. Vehicle 101 mayalso include a status indicating apparatus, such as status bar 230, toindicate the current status of vehicle 101. In the example of FIG. 2,status bar 230 displays “D” and “2 mph” indicating that the vehicle ispresently in drive mode and is moving at 2 miles per hour. In thatregard, the vehicle may display text on an electronic display,illuminate portions of vehicle 101, or provide various other types ofindications. In addition, the computer may also have external indicatorswhich indicate whether, at the moment, a human or an automated system isin control of the vehicle, that are readable by humans, other computers,or both.

In one example, computer 110 may be an autonomous driving computingsystem capable of communicating with various components of the vehicle.For example, computer 110 may be in communication with the vehicle'sconventional central processor 160, and may send and receive informationfrom the various systems of vehicle 101, for example the braking 180,acceleration 182, signaling 184, and navigation 186 systems in order tocontrol the movement, speed, etc. of vehicle 101. In addition, whenengaged, computer 110 may control some or all of these functions ofvehicle 101 and thus be fully or merely partially autonomous. It will beunderstood that although various systems and computer 110 are shownwithin vehicle 101, these elements may be external to vehicle 101 orphysically separated by large distances.

FIG. 2 depicts an exemplary design of the interior of an autonomousvehicle. The autonomous vehicle may include all of the features of anon-autonomous vehicle, for example: a steering apparatus, such assteering wheel 210; a navigation display apparatus, such as navigationdisplay 215; and a gear selector apparatus, such as gear shifter 220.

Vehicle 101 may include one or more user input devices that enable auser to provide information to the autonomous driving computer 110. Forexample, a user, such as passenger 290, may input a destination (e.g.,123 Oak Street) into the navigation system using touch screen 217 orbutton inputs 219. In another example, a user may input a destination byidentifying the destination. In that regard, the computer system mayextract the destination from a user's spoken command.

The vehicle may also have various user input devices for activating ordeactivating one or more autonomous driving modes. In some examples, thedriver may take control of the vehicle from the computer system byturning the steering wheel, or pressing the acceleration or decelerationpedals. The vehicle may further include a large emergency button thatdiscontinues all or nearly all of the computer's decision-making controlrelating to the car's velocity or direction. In another example, thevehicle's shift knob may be used to activate, adjust, or deactivate theautonomous modes.

Computer 110 may include, or be capable of receiving information from,one or more touch sensitive input apparatuses 140. For example, computer110 may receive input from a user input apparatus and use thisinformation to determine whether a passenger is in contact with, such asby holding or bumping, a particular portion of vehicle 110. The touchsensitive input apparatuses may be any touch sensitive input devicecapable of identifying a force, for example a force-resistance tape maybe calibrated to accept or identify a threshold pressure input (such as10 grams of pressure) or a range of pressures (such as 5-20 grams ofpressure).

These inputs may be understood by the computer as commands from the userto, for example, enter into or exit from one or more autonomous drivingmodes. For example, if the vehicle is being operated in an autonomousmode and the driver bumps the steering wheel, if the force is above thethreshold input, the vehicle may exit an autonomous mode and enter asemi-autonomous mode where the driver has control of at least thesteering.

The various systems described above may be used by the computer tooperate the vehicle and maneuver from one location to another. Forexample, a user may enter destination information into a navigationsystem, either manually or audibly. The vehicle may determine itslocation within a few inches based on a combination of the GPS receiverdata and the sensor data, as well as the detailed map information. Inresponse, the navigation system may generate a route between the presentlocation of the vehicle and the destination.

When the driver is ready to relinquish some level of control to theautonomous driving computer, the user may activate computer control. Thecomputer may be activated, for example, by pressing a button or bymanipulating a lever such as gear shifter 220. Rather than takingcontrol immediately, the computer may scan the surroundings anddetermine whether there are any obstacles or objects in the immediatevicinity which may prohibit or reduce the ability of the vehicle toavoid a collision. In this regard, the computer may require that thedriver continue controlling the vehicle manually or with some level ofcontrol (such as the steering or acceleration) before entering into afully autonomous mode.

Once the vehicle is able to maneuver safely without the assistance ofthe driver, the vehicle may become fully autonomous and continue to thedestination. The driver may continue to assist the vehicle bycontrolling, for example, steering or whether the vehicle changes lanes,or the driver may take control of the vehicle immediately in the eventof an emergency.

The vehicle may continuously use the sensor data to identify objects,such as traffic signals, people, other vehicles, and other objects, inorder to maneuver the vehicle to the destination and reduce thelikelihood of a collision. The vehicle may use the map data to determinewhere traffic signals or other objects may appear and respond by, forexample, signaling to turn or change lanes.

Once the vehicle has arrived at the destination, the vehicle may provideaudible or visual cues to the driver. For example, by displaying “Youhave arrived” on one or more of the electronic displays.

FIG. 3 illustrates an exemplary system diagram in accordance withaspects of the disclosure. As shown in scenario 300, the autonomousdriving computer system 110 may continuously collect and utilize varioustypes of real-time and non-real time data (such as data 302, 304, 306and 308) to generate (by recording or deriving) user-specific drivingpatterns and perform autonomous controlling of the vehicle based on thegenerated user-specific driving patterns. For example, the autonomousdriving computer system 110 may include a user interface system 312 toreceive user input data 302. The user interface system 312 may be anykind of input device for a computer system, e.g., an electronic displaysuch as a touch-screen, a keypad and/or microphone. As such, variousprompts for user information may be displayed or otherwise communicatedto the driver, and the driver may enter any desired input through theinterface system 312. For example, user data input 302 may include useridentification (e.g., the driver's name or a reference number) and/orother user attribute data such as gender and age. User input 302 mayalso include various user preference data such as preference for warningof any vehicle in an adjacent lane within a certain distance, making oneor more specific stops along a route to a destination, etc.

The autonomous driving computer system 310 may also include a datacollection and processing system 314, which may include various types ofsensors and information gathering devices or systems as previouslydiscussed for collecting object data 304, environmental data 306 andvehicle data 308. For example, system 314 may include radar, opticalsensors, ultrasonic sensors, active and passive infrared sensors, radiofrequency sensors, and cameras. System 314 may also include GPS andAssisted GPS systems to receive location signals, and communicationsystems such as wireless transmitters and receivers to communicate withremote servers or other types of information sources. The datacollection and processing system 314 may also be external and connect tothe computer system 310 directly, wirelessly or through various kinds ofcommunication infrastructures available on the vehicle.

Object data 304 may include various types of moving or static objectswithin the sensors' respective sensing ranges. For example, object data304 may include information about moving objects such as vehicles andpedestrians, and information about static objects such as roads, paths,lanes, buildings, signs, etc. The information may include relativevelocity and distance to the autonomous vehicle, and other parametersfor identifying and describing the objects.

Environmental data 306 may include information such as trafficconditions, weather conditions and road conditions etc. Environmentaldata 306 may be detected by the various types of sensors as describedabove or may be acquired through various communication devices fromoutside sources (e.g., servers providing weather, visibility or othertypes of driving condition information).

Vehicle data 308 may include various types of performance or operationaldata related to the vehicle, for example, brake/throttle applications,acceleration, velocity, wheel speed, lane changes, distances to thevehicles to the front and rear, tire pressure, suspension data, steeringangles, engine temperature, gas usage, etc. Vehicle data 308 may alsoinclude vehicle location data and cargo loading data. Besides theexternal data, vehicle data 308 may also include other signalsindicating the internal conditions of the vehicle, for example, weightmeasurements and distribution of the occupants (driver and passengers).

The autonomous vehicle may continuously monitor the vehicle and theobjects moving in the vicinity of the vehicle, and collect theaforementioned data. Using such data collected over time, together withthe user input data 302 and various types of non-real time data 316(e.g., a detailed map) stored in the computer system, a machine learningsystem 318 is able to derive various driver preferences and generatedriver specific patterns 320. For example, a particular driver's patternmay indicate a driver's preference of windy roads over straight roads,right turns over left turns, multi-lane highways over side roads, etc.In another example, a driver pattern may also include parametersgeneralizing how the driver normally changes lanes, enters onto or exitsfrom a highway, starts from a traffic stop, and maintains or changes thespace between vehicles in the front or to the rear.

From these patterns, the autonomous driving computer system may predicthow a specific driver will drive. Various machine learning techniquesand algorithms may be used to facilitate the learning process anddriving predictions. For example, the vehicle may include a “record”button to put the vehicle into a training mode to record the actions ofthe driver. The driver may drive the vehicle for a day while the vehiclemonitors how much torque is applied to the steering wheel, how quicklythe driver accelerates at an interaction or highway, whether or not thedriver selects to change lanes and pass slower vehicles, and how thedriver applies the brakes. The vehicle may then learn the driver'spreferred style and pattern, and replicate the pattern when the driveruses the vehicle in an autonomous mode.

The learning process may also record a specific route that the driverfollows each day as well as the driver's style during the route. Then,the driver may select a “play” button and replay the route on the userdisplay. Alternatively, the driver may select a “repeat trip” button,causing the vehicle to follow the same route making similar choices,though making alterations as necessary for safety, as the driver haddone during the recording mode.

The learning period may depend on various factors such as the specificdriving feature that the autonomous driving computer system 310 tries tolearn. By way of example, the preference of multi-lane highways overside roads may take a longer time to learn than the preference ofacceleration speed. As such, system 310 may adopt different combinationsof machine learning algorithms with regard to different drivingfeatures.

Using the learned driving patterns, the vehicle may perform driverspecific autonomous driving when it identifies which driver is currentlyin the driver's seat. The system may control the autonomous drivingunder the limits of the one or more safe envelope driving patterns 322.For instance, each autonomous vehicle may be configured with a set ofdefault general safe envelope driving patterns that are applicable toall the drivers. The safe envelope driving patterns may be one or moresets of parameters limiting how a driver should drive, for example, bycontrolling the speed of changing lanes and the pattern of passing byvehicles, etc. The safe envelope patterns may also be set differentlyfor different driver categories, or may be customized or adjusted byeach individual driver. For example, the highest allowable speed on ahighway or in making a turn may be set lower for users within aparticular age group. In another example, certain routes may bedisallowed under certain weather conditions in a safe envelope for aparticular driver but allowed in another safe envelope for anotherdriver.

As such, the autonomous vehicle may be driven under a set ofdriver-specific control commands 326, which may be generated by anavigation controller 324 based on both the safe envelopes 322 and thedriver specific patterns 320. Thus, the autonomous vehicle is able toperform adaptive autonomous driving based on driver preferences butstill safely within the general safe envelope driving patterns. Forexample, when the vehicle detects user A is the current driver, thecontrol commands 326 may be generated to control the vehicle to changelanes more often in order to move faster for driver A than for driver B,but the overall speed and the frequency of changing lanes are all stillwithin the safe envelope boundary. In another example, the autonomousvehicle may select a route with more windy roads for driver C and aroute with more straight roads for driver D, but the selected routes areall allowable by the safe envelope or are not among the disallowedroutes of the safe envelope. In a further example, the autonomousvehicle may follow closer to the vehicle in front for a driver E thanfor driver F, who is less aggressive than driver E, but still keep thedistance to the vehicle in front above the minimum distance designatedin the safe envelope.

The autonomous vehicle may also take various counter measures to ensurethe driving is within the safe envelope. The countermeasures mayinclude, but is not limited to, brake control, throttle control,steering control, transmission control and suspension control.

Operations in accordance with aspects of the disclosure will now bedescribed with reference to FIGS. 4-5. It should be understood that thefollowing operations do not have to be performed in the precise orderdescribed below. Rather, various operations can be handled in adifferent order or simultaneously. It should also be understood thatthese operations do not have to be performed all at once. For instance,some operations may be performed separately from other operations.

FIG. 4 illustrates a flow chart 400 where an autonomous vehicle collectsdata and builds user specific driving patterns. As shown in FIG. 4, thedriving pattern recognition process 400 preferably starts in block 402with the autonomous driving computer system obtaining user input datasuch as user identification, preferences and/or other informationindicating a user's profile. A user's identification may include auser's name or a reference number. User preference data may include dataindicating, for example, a preference for quicker acceleration, apreference for frequent stops along a route, or a preference formaintaining a speed at the legal speed limit. The autonomous drivingcomputer system may also receive information indicating a user'sprofile, for example, experienced vs. inexperienced, and levels of risksor aggressiveness that the user is willing to accept.

The autonomous driving computer system may start, at the same time andin block 404, to continuously collect data from various sensor systemsand outside sources regarding the vehicle's operations/conditions,objects in the vicinity of the vehicle, traffic, weather and roadconditions. The data may be collected by, for example, GPS, inertialsensors, lasers, radar, sonar, and acoustic sensors. Other feedbacksignals, such as input from sensors typical of non-autonomous vehicles,may also be read from the vehicle. Example sensors typical ofnon-autonomous vehicles include tire pressure sensors, enginetemperature sensors, brake heat sensors, brake pad status sensors, tiretread sensors, fuel sensors, oil level and quality sensors, air qualitysensors (for detecting temperature, humidity, or particulates in theair), etc. In block 406, the system may also retrieve various types ofnon-real time data (e.g., a detailed map) stored in the system or fromthe remote information sources.

In block 408, the autonomous driving computer system may determine oneor more driver specific patterns based on the received useridentification, the data collected in real time and the non-real timedata. The system may also build profiles for each individual driver tofacilitate storage of the data in local memory or remote storage andretrieval by authorized users. Each profile includes the drivingpatterns and other user-related information.

FIG. 5 illustrates a process 500 where the autonomous driving systemoperates the vehicle based on the driver-specific patterns. In block502, the system identifies the user sitting in the driver's seat by, forexample, the driver logging-in to the system or other typical userauthentication techniques. In block 504, the system retrieves thedriver-specific driving pattern based on the user's identification. Inblock 506, the system also obtains safe envelope driving patterns fromlocal or remote memory. In block 508, the system collects data fromvarious sensor systems about the operations and conditions of thevehicle, and the objects surrounding the vehicle. The system alsoreceives other information such as traffic and weather information fromother sources.

In block 510, the vehicle generates control commands based on the safeenvelope driving patterns, the driver specific patterns and the datacollected in block 508, and performs autonomous driving accordingly. Inblock 512, the autonomous driving computer system continually monitorswhether the autonomous driving is interrupted by the driver. Ifautonomous driving is interrupted, the system starts to control thevehicle in response to the user interruption in block 514. Otherwise,the system continues collecting sensor data and controlling the vehiclein an autonomous mode.

Systems and methods according to aspects of the disclosure are notlimited to detecting any particular type of objects or observing anyspecific type of vehicle operations or environmental conditions, norlimited to any particular machine learning algorithm, but may be usedfor deriving and learning any driving pattern with any unique signatureto be differentiated from other driving patterns.

The sample values, types and configurations of data described and shownin the figures are for the purposes of illustration only. In thatregard, systems and methods in accordance with aspects of the disclosuremay include various types of sensors, communication devices, userinterfaces, vehicle control systems, data values, data types andconfigurations. The systems and methods may be provided and received atdifferent times (e.g., via different servers or databases) and bydifferent entities (e.g., some values may be pre-suggested or providedfrom different sources).

As these and other variations and combinations of the features discussedabove can be utilized without departing from the systems and methods asdefined by the claims, the foregoing description of exemplaryembodiments should be taken by way of illustration rather than by way oflimitation of the disclosure as defined by the claims. It will also beunderstood that the provision of examples (as well as clauses phrased as“such as,” “e.g.”, “including” and the like) should not be interpretedas limiting the disclosure to the specific examples; rather, theexamples are intended to illustrate only some of many possible aspects.

Unless expressly stated to the contrary, every feature in a givenembodiment, alternative or example may be used in any other embodiment,alternative or example herein. For instance, any appropriate sensor fordetecting vehicle movements may be employed in any configuration herein.Any data structure for representing a specific driver pattern or asignature vehicle movement may be employed. Any suitable machinelearning algorithms may be used with any of the configurations herein.

The invention claimed is:
 1. A method for controlling a vehicle, themethod comprising: determining, by a processor, a set of safe drivingrequirements to control the vehicle in an autonomous mode, the set ofsafe driving requirements defining a lane changing limit; receiving useridentification data; receiving sensor data from one or more sensorsassociated with the vehicle; determining, by the processor, a driverspecific pattern based on the received sensor data and the received useridentification data; and controlling, by the processor, operation of thevehicle in the autonomous mode based on the driver specific pattern andthe lane changing limit of the set of safe driving requirements.
 2. Themethod of claim 1, wherein the set of safe driving requirementsindicates a minimum distance from the vehicle to a different vehiclepositioned in front of the vehicle.
 3. The method of claim 2, whereinthe received sensor data comprises data indicating an average distanceto the different vehicle in front of the vehicle and changes in thedistance over a time period.
 4. The method of claim 3, whereincontrolling operation of the vehicle further includes maintaining adistance within a range of the average distance and greater than aminimum distance.
 5. The method of claim 1, wherein the lane changinglimit indicates a frequency of changing lanes.
 6. The method of claim 1,further comprising receiving information indicating a choice of a routefrom a starting point to a destination point and wherein controllingoperation of the vehicle is further based on the received information.7. The method of claim 1, wherein determining the river specific patternfurther comprises processing the received sensor data using machinelearning and wherein controlling the vehicle is further based on theprocessed received sensor data.
 8. The method of claim 1, whereincontrolling operation of the vehicle comprises controlling the vehiclein accordance with the set of safe driving requirements when there areconflicts between the set of safe driving requirements and the driverspecific pattern.
 9. The method of claim 1, further comprising:receiving a user preference of operation of the vehicle; and determiningthe driver specific pattern based on the received user preference, thereceived sensor data, and the received user identification data.
 10. Themethod of claim 9, wherein the user preference indicates a range ofacceleration speed.
 11. The method of claim 1, further comprisingreceiving a user interruption and controlling the vehicle based on thereceived user interruption.
 12. A device for controlling a vehicle withan autonomous driving mode, the device comprising: one or more sensorsfor detecting information surrounding the vehicle; a processor coupledto the one or more sensors; memory coupled to the processor and storinginstructions executable by the processor causing the processor to:determine a set of safe driving requirements to control the vehicle inan autonomous mode, the set of safe driving requirements defining a lanechanging limit; receive user identification data; receive sensor datafrom one or more sensors associated with the vehicle; determine a driverspecific pattern based on the received sensor data and the received useridentification data; and control operation of the vehicle in theautonomous mode based on the driver specific pattern, and the limit ofthe set of safe driving requirements.
 13. The device of claim 12,wherein the set of safe driving requirements indicates a minimumdistance from the vehicle to a different vehicle positioned in front ofthe vehicle.
 14. The device of claim 13, wherein the received sensordata comprises data indicating an average distance to the differentvehicle in front of the vehicle and changes in the distance over a timeperiod.
 15. The device of claim 14, wherein controlling operation of thevehicle further includes maintaining a distance within a range of theaverage distance and greater than a minimum distance.
 16. The device ofclaim 12, wherein the received information comprises data indicating afrequency of changing lanes.
 17. The device of claim 12, wherein thereceived information comprises data indicating a choice of a route froma starting point to a destination point.
 18. The device of claim 12,wherein the driver specific pattern is determined by processing thereceived sensor data using machine learning.
 19. The device of claim 12,wherein control of operation of the vehicle comprises controlling thevehicle in accordance with the set of safe driving requirements whenthere are conflicts between the set of safe driving requirements and thedriver specific pattern.
 20. The device of claim 12, wherein additionalinstructions executable by the processor cause the processor to: receivea user preference of operation of the vehicle; and determine the driverspecific pattern based on the received user preference, the receivedsensor data, and the received user identification data.
 21. The deviceof claim 20, wherein the user preference indicates a range ofacceleration speed.
 22. The device of claim 12, wherein additionalinstructions executable by the processor allow the processor to detect auser interruption and control the vehicle based on the received userinterruption.